Learning Speech Emotion Representations in the Quaternion Domain

The modeling of human emotion expression in speech signals is an important, yet challenging task. high resource demand recognition models, combined with the general scarcity emotion-labelled data are obstacles to development and application effective solutions this field. In paper, we present approach jointly circumvent these difficulties. Our method, named RH-emo, a novel semi-supervised architecture aimed at extracting quaternion embeddings from real-valued monoaural spectrograms, enabling use quaternion-valued networks for tasks. RH-emo hybrid real/quaternion autoencoder network that consists encoder parallel classifier decoder. On one hand, permits optimization each latent axis classification specific emotion-related characteristic: valence, arousal, dominance, overall emotion. other reconstruction enables dimension develop intra-channel correlations required representation as entity. We test our on tasks using four popular datasets: IEMOCAP, RAVDESS, EmoDB, TESS, comparing performance three well-established CNN architectures (AlexNet, ResNet-50, VGG) their equivalent fed created RH-emo. obtain consistent improvement accuracy all datasets, while drastically reducing resources' models. Moreover, performed additional experiments ablation studies confirm effectiveness approach. repository available at: https://github.com/ispamm/rhemo .